Metascheduling and Heuristic Co-Allocation Strategies in Distributed Computing

In this paper, we address problems of efficient computing in distributed systems with non-dedicated resources including utility grid. There are global job flows from external users along with resource owner's local tasks upon the resource non-dedication condition. Competition for resource reservation between independent users, local and global job flows substantially complicates scheduling and the requirement to provide the necessary quality of service. A metascheduling concept, justified in this work, assumes a complex combination of job flow dispatching and application-level scheduling methods for parallel jobs, as well as resource sharing and consumption policies established in virtual organizations and based on economic principles. We introduce heuristic slot selection and co-allocation strategies for parallel jobs. They are formalized by given criteria and implemented by algorithms of linear complexity on an available slots number

Metascheduling Strategies in Distributed Computing with Non-dedicated Resources

In this chapter, we address problems of efficient computing in distributed systems with non-dedicated resources including utility Grid. There are global job flows from external users along with resource owner’s local tasks upon resource non-dedication condition. Competition for resource reservation between independent users, local and global job flows substantially complicates scheduling and the requirement to provide the necessary quality of service. A metascheduling concept, justified in this work, assumes a complex combination of job flow dispatching and application-level scheduling methods for parallel jobs, as well as resource sharing and consumption policies established in virtual organizations and based on economic principles

Job Flow Distribution and Ranked Jobs Scheduling in Grid Virtual Organizations

In this work, we consider the problems of job flow distribution and ranked job framework forming within a model of cycle scheduling in Grid virtual organizations. The problem of job flow distribution is solved in terms of jobs and computing resource domains compatibility. A coefficient estimating such compatibility is introduced and studied experimentally. Two distribution strategies are suggested. Job framework forming is justified with such quality of service indicators as an average job execution time, a number of required scheduling cycles, and a number of job execution declines. Two methods for job selection and scheduling are proposed and compared: the first one is based on the knapsack problem solution, while the second one utilizes the mentioned compatibility coefficient. Along with these methods we present experimental results demonstrating the efficiency of proposed approaches and compare them with random job selection

Preference-based Fair Resource Sharing and Scheduling Optimization in Grid VOs

AbstractIn this paper, we deal with problems of efficient resource management and scheduling in utility Grids. There are global job flows from external users along with resource owners’ local tasks upon resource non-dedication condition. Competition for resource reservation between independent users, local and global job flows substantially complicates scheduling and the requirement to provide the necessary quality of service. A meta-scheduling model, justified in this work, assumes a complex combination of job flow dispatching and application-level scheduling methods for jobs, as well as resource sharing and consumption policies established in virtual organizations (VOs) and based on economic principles. A solution to the problem of fair resource sharing among VO stakeholders with simulation studies is proposed

Deep Underground Neutrino Experiment (DUNE), Far Detector Technical Design Report, Volume I Introduction to DUNE

International audienceThe preponderance of matter over antimatter in the early universe, the dynamics of the supernovae that produced the heavy elements necessary for life, and whether protons eventually decay—these mysteries at the forefront of particle physics and astrophysics are key to understanding the early evolution of our universe, its current state, and its eventual fate. The Deep Underground Neutrino Experiment (DUNE) is an international world-class experiment dedicated to addressing these questions as it searches for leptonic charge-parity symmetry violation, stands ready to capture supernova neutrino bursts, and seeks to observe nucleon decay as a signature of a grand unified theory underlying the standard model. The DUNE far detector technical design report (TDR) describes the DUNE physics program and the technical designs of the single- and dual-phase DUNE liquid argon TPC far detector modules. This TDR is intended to justify the technical choices for the far detector that flow down from the high-level physics goals through requirements at all levels of the Project. Volume I contains an executive summary that introduces the DUNE science program, the far detector and the strategy for its modular designs, and the organization and management of the Project. The remainder of Volume I provides more detail on the science program that drives the choice of detector technologies and on the technologies themselves. It also introduces the designs for the DUNE near detector and the DUNE computing model, for which DUNE is planning design reports. Volume II of this TDR describes DUNE's physics program in detail. Volume III describes the technical coordination required for the far detector design, construction, installation, and integration, and its organizational structure. Volume IV describes the single-phase far detector technology. A planned Volume V will describe the dual-phase technology

Deep Underground Neutrino Experiment (DUNE), Far Detector Technical Design Report, Volume II: DUNE Physics

The preponderance of matter over antimatter in the early universe, the dynamics of the supernovae that produced the heavy elements necessary for life, and whether protons eventually decay -- these mysteries at the forefront of particle physics and astrophysics are key to understanding the early evolution of our universe, its current state, and its eventual fate. DUNE is an international world-class experiment dedicated to addressing these questions as it searches for leptonic charge-parity symmetry violation, stands ready to capture supernova neutrino bursts, and seeks to observe nucleon decay as a signature of a grand unified theory underlying the standard model. The DUNE far detector technical design report (TDR) describes the DUNE physics program and the technical designs of the single- and dual-phase DUNE liquid argon TPC far detector modules. Volume II of this TDR, DUNE Physics, describes the array of identified scientific opportunities and key goals. Crucially, we also report our best current understanding of the capability of DUNE to realize these goals, along with the detailed arguments and investigations on which this understanding is based. This TDR volume documents the scientific basis underlying the conception and design of the LBNF/DUNE experimental configurations. As a result, the description of DUNE's experimental capabilities constitutes the bulk of the document. Key linkages between requirements for successful execution of the physics program and primary specifications of the experimental configurations are drawn and summarized. This document also serves a wider purpose as a statement on the scientific potential of DUNE as a central component within a global program of frontier theoretical and experimental particle physics research. Thus, the presentation also aims to serve as a resource for the particle physics community at large

Deep Underground Neutrino Experiment (DUNE) Near Detector Conceptual Design Report

International audienceThe Deep Underground Neutrino Experiment (DUNE) is an international, world-class experiment aimed at exploring fundamental questions about the universe that are at the forefront of astrophysics and particle physics research. DUNE will study questions pertaining to the preponderance of matter over antimatter in the early universe, the dynamics of supernovae, the subtleties of neutrino interaction physics, and a number of beyond the Standard Model topics accessible in a powerful neutrino beam. A critical component of the DUNE physics program involves the study of changes in a powerful beam of neutrinos, i.e., neutrino oscillations, as the neutrinos propagate a long distance. The experiment consists of a near detector, sited close to the source of the beam, and a far detector, sited along the beam at a large distance. This document, the DUNE Near Detector Conceptual Design Report (CDR), describes the design of the DUNE near detector and the science program that drives the design and technology choices. The goals and requirements underlying the design, along with projected performance are given. It serves as a starting point for a more detailed design that will be described in future documents

Scintillation light detection in the 6-m drift-length ProtoDUNE Dual Phase liquid argon TPC

DUNE is a dual-site experiment for long-baseline neutrino oscillation studies, neutrino astrophysics and nucleon decay searches. ProtoDUNE Dual Phase (DP) is a 6 $\times$ 6 $\times$ 6 m$^3$ liquid argon time-projection-chamber (LArTPC) that recorded cosmic-muon data at the CERN Neutrino Platform in 2019–2020 as a prototype of the DUNE Far Detector. Charged particles propagating through the LArTPC produce ionization and scintillation light. The scintillation light signal in these detectors can provide the trigger for non-beam events. In addition, it adds precise timing capabilities and improves the calorimetry measurements. In ProtoDUNE-DP, scintillation and electroluminescence light produced by cosmic muons in the LArTPC is collected by photomultiplier tubes placed up to 7 m away from the ionizing track. In this paper, the ProtoDUNE-DP photon detection system performance is evaluated with a particular focus on the different wavelength shifters, such as PEN and TPB, and the use of Xe-doped LAr, considering its future use in giant LArTPCs. The scintillation light production and propagation processes are analyzed and a comparison of simulation to data is performed, improving understanding of the liquid argon properties.DUNE is a dual-site experiment for long-baseline neutrino oscillation studies, neutrino astrophysics and nucleon decay searches. ProtoDUNE Dual Phase (DP) is a 6x6x6m3 liquid argon time-projection-chamber (LArTPC) that recorded cosmic-muon data at the CERN Neutrino Platform in 2019-2020 as a prototype of the DUNE Far Detector. Charged particles propagating through the LArTPC produce ionization and scintillation light. The scintillation light signal in these detectors can provide the trigger for non-beam events. In addition, it adds precise timing capabilities and improves the calorimetry measurements. In ProtoDUNE-DP, scintillation and electroluminescence light produced by cosmic muons in the LArTPC is collected by photomultiplier tubes placed up to 7 m away from the ionizing track. In this paper, the ProtoDUNE-DP photon detection system performance is evaluated with a particular focus on the different wavelength shifters, such as PEN and TPB, and the use of Xe-doped LAr, considering its future use in giant LArTPCs. The scintillation light production and propagation processes are analyzed and a comparison of simulation to data is performed, improving understanding of the liquid argon properties

DUNE Offline Computing Conceptual Design Report

This document describes Offline Software and Computing for the Deep Underground Neutrino Experiment (DUNE) experiment, in particular, the conceptual design of the offline computing needed to accomplish its physics goals. Our emphasis in this document is the development of the computing infrastructure needed to acquire, catalog, reconstruct, simulate and analyze the data from the DUNE experiment and its prototypes. In this effort, we concentrate on developing the tools and systems thatfacilitate the development and deployment of advanced algorithms. Rather than prescribing particular algorithms, our goal is to provide resources that are flexible and accessible enough to support creative software solutions as HEP computing evolves and to provide computing that achieves the physics goals of the DUNE experiment

Reconstruction of interactions in the ProtoDUNE-SP detector with Pandora

International audienceThe Pandora Software Development Kit and algorithm libraries provide pattern-recognition logic essential to the reconstruction of particle interactions in liquid argon time projection chamber detectors. Pandora is the primary event reconstruction software used at ProtoDUNE-SP, a prototype for the Deep Underground Neutrino Experiment far detector. ProtoDUNE-SP, located at CERN, is exposed to a charged-particle test beam. This paper gives an overview of the Pandora reconstruction algorithms and how they have been tailored for use at ProtoDUNE-SP. In complex events with numerous cosmic-ray and beam background particles, the simulated reconstruction and identification efficiency for triggered test-beam particles is above 80% for the majority of particle type and beam momentum combinations. Specifically, simulated 1 GeV/$c$ charged pions and protons are correctly reconstructed and identified with efficiencies of 86.1$\pm0.6$% and 84.1$\pm0.6$%, respectively. The efficiencies measured for test-beam data are shown to be within 5% of those predicted by the simulation